Method and system for capturing images for wound assessment with moisture detection

ABSTRACT

A wound assessment method which can estimate a moisture level of the wound, and related image capture device. The wound area is imaged at least twice where the wound is illuminated under different illumination light intensities. The first image captured using a relatively low illumination light intensity is analyzed to assess the wound, for example measuring its size, color and texture. The second image captures using a relatively high illumination light intensity (e.g. using a flash) is analyzed to estimate the moisture level of the wound. The moisture level estimation method extracts white connected components from the second image, and estimates the moisture level based on the number, sizes, and centroid distribution of the white connected components. A 3D image of the wound may also be captured, e.g. using a structured-light 3D scanner of the image capture device.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a device and method for capturingimages of visible injuries or wounds and for performing woundassessment, and more particularly, it relates to wound assessment withmoisture level detection.

Description of Related Art

For many wound types, such as pressure ulcers, recovery times can bevery long. To track wound progress and perform proper treatment, thefirst step is to capture images of the wound properly. Wound images maythen be analyzed to measure the size, texture and color of the wound.One useful aspect of wound assessment is estimating the moisture levelof a wound. Currently there is no solution to estimate the moisturelevel of a wound. Also, when a wound has a high moisture level due toblood or pus, it may have a wet and thus shiny wound surface, whichmakes the wound difficult to properly image; the measurement of a woundwith 2D and 3D cameras in such a case is often not stable.

SUMMARY

Accordingly, the present invention is directed to a method for capturingimages of a wound that substantially obviates one or more of theproblems due to limitations and disadvantages of the related art.

An object of the present invention is to provide a method for properlycapturing wound images and measuring the wound dimension and moisturelevel using an image capturing device alone without using additionalmeasuring devices.

Additional features and advantages of the invention will be set forth inthe descriptions that follow and in part will be apparent from thedescription, or may be learned by practice of the invention. Theobjectives and other advantages of the invention will be realized andattained by the structure particularly pointed out in the writtendescription and claims thereof as well as the appended drawings.

To achieve these and/or other objects, as embodied and broadlydescribed, the present invention provides a method implemented in asystem including an image capture device for assessing a wound of apatient, which includes: (a) capturing a first image of a part of thepatient's body which contains the wound, the part of the patient's bodybeing illuminated by a light of a first intensity during the capture;(b) capturing a second image of the part of the patient's body, the partof the patient's body being illuminated by a light of a second intensityduring the capture, the second intensity being stronger than the firstintensity; (c) estimating a moisture level of the wound by analyzing thesecond image; (d) assessing the wound by analyzing the first image,including measuring at least one of a size, texture and color of thewound; and (e) registering assessment results of the wound, wherein theassessment result includes the estimated moisture level and the at leastone of the size, texture and color of the wound.

In a preferred embodiment, the step of estimating a moisture level ofthe wound includes extracting white connected components in the woundarea, and estimating the moisture level of the wound based on a numberof the white connected components, sizes of the white connectedcomponents, and a distribution of locations of the white connectedcomponents.

In another aspect, the present invention provides an image capturedevice which includes: a 2D camera for capturing two-dimensional images;a 3D camera for capturing three-dimensional images; a user interfacescreen for displaying images and information and for interacting with auser; a processor; and a computer usable non-transitory memory having acomputer readable program code embedded therein which is configured tobe executed by the processor to control the 2D camera, the 3D camera,and the user interface, and to perform the above described process ofassessing a wound of a patient.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and areintended to provide further explanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustration of an imaging device for wound assessment witha graphical user interface in accordance with an exemplary embodiment.

FIG. 2 is a schematic hardware block diagram showing the structure ofthe imaging device of FIG. 1.

FIG. 3 schematically illustrates a wound image capture and woundassessment process according to a first embodiment of the presentinvention.

FIGS. 4 and 5 schematically illustrate the details of some of the stepsin the process of FIG. 3.

FIG. 6 schematically illustrates a wound image capture and woundassessment process according to a second embodiment of the presentinvention.

FIG. 7 schematically illustrates a wound image capture and woundassessment process according to a third embodiment of the presentinvention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

FIG. 1 is an illustration of an imaging device 1 for wound assessmentwith a graphical user interface according to an embodiment of thepresent invention. FIG. 2 is a hardware block diagram showing thestructure of the imaging device 1. As shown in FIG. 1, the device 1includes a graphical user interface or screen 15, which preferablyincludes a touch screen or panel 16 on a front side. The device 1 alsoincludes one or more cameras 13, 14, preferably in the form of atwo-dimensional (2D camera) 13 and a three-dimensional (3D) camera 14 ona back side. For example, the imaging device 1 may be a tablet computer,for example, an iPhone®, an iPad®, Android enabled devices and/or othervarious handheld mobile devise, which includes one or more camerashaving the ability to capture and generate 3D images. As shown in FIG.1, the device 1 may be a tablet or tablet-like device having the userinterface or display 15 and an outer housing 30.

Electrical components are positioned within the housing. The electroniccomponents may vary depending on the particular functionality of thedevice 1. However, by way of example, the electronic components mayinclude, for example, a communication interface 17, a processor 11, anda memory 12 storing computer executable programs. The imaging device 1may communicate with a data processing and storage server 2 via anetwork 19. The method described below may be implemented by computerexecutable programs stored on the imaging device 1 and/or or the server2 executed by the processor of the imaging device and/or the server.

During wound image capture in preferred embodiments of the presentinvention, both the 2D and 3D cameras are used to capture image data.For example, 3D images may be used to estimate the depth of the wound,and 2D images may be used to measure the size and color of the wound.

In a preferred embodiment, the 3D camera 14 is a structured-light 3Dscanner that uses structured-light 3D scanning methods to obtain 3D dataabout objects. This is accomplished by projecting a pattern of light(e.g., visible or IR light) from the scanner onto the 3D surface of theobject, and capturing the image (e.g., a video) of the light pattern onthe object's surface using one or more sensors that have differentperspectives than the light projector. Due to the shape of the 3D objectand the different perspectives, the light pattern in the captured imageswill appear distorted as compared to the original projected pattern;such information is used to exact geometric information of the objectsurface. These 3D scanners are fast; some can scan an entire field ofview in less a second. Structured-light 3D scanners are availablecommercially. Any suitable structured-light 3D scanners may be used toimplement embodiments of the present invention.

The structured-light 3D scanner used in embodiments of the invention canautomatically determine the distance of the object at the center of theview in order to properly adjust the setting of its sensors and/oroptics to successfully capture the 3D data. However, the inventorsdiscovered that when the wound located at the center of the view has ashiny surface, the 3D scanner often fails to determine the distance andtherefore unable to properly capture the 3D data.

Embodiments of the present invention provide a wound image capturemethod that achieves two goals: First, to measure moisture level ofwound as a wound assessment value; and second, to improve the quality ofthe captured images, including 3D images, which are used to assess otheraspects of the wound. Image capture methods according to the presentembodiments use the responses of the cameras to the differentintensities of the incident illumination light to estimate the moisturelevel of a wound, in the meantime to help obtain correct woundmeasurement even when the wound is wet due to blood or pus.

FIG. 3 schematically illustrates a wound image capture and woundassessment process according to a first embodiment of the presentinvention. As shown in FIG. 3, at the beginning of the process, the useruses the 3D camera to capture a 3D image of the wound (step S104). Thissteps includes repeatedly attempting the 3D image capture until thecamera can properly determine the distance of the object and capture the3D image; the number of failures in that process is recorded. This stepis performed with a first illumination light intensity (step S102). Theillumination light is provided by a light source of the image capturedevice 1, for example, a light source 13 a associated with the 2D camera13. The first light intensity is preferably a medium or default lightintensity level of the light source 13 a, or is lower than the medium ordefault level. Alternatively, an additional light source can be attachedto the image capture device 1 or used separately.

In the preferred embodiment, the 3D scanner 14 is controlled by itsinternal programs so that when 3D data capture is unsuccessful due toinability to determine the distance of the object or other reasons, itautomatically repeats the image capture procedure. The 3D scanner willcontinue to attempt to capture 3D data while displaying messages orother types of indication on the user interface (the preview screen) toindicate the status of 3D data capture, for example to indicate thatcapture is in progress, or that a capture attempt did not succeed, orthat a capture has been successful, or to suggest that the user movesthe aim of the 3D scanner slightly, etc.

In operation of step S104, as shown in FIG. 4, the user initiates the 3Dscanning mode of the image capture device 1, holds the device and aimsthe 3D camera at a part of the patient's body that contains the wound,placing the wound at the center of the preview screen, and presses a“scan” or “start” button on the user interface (e.g. touch screen) (stepS402). The image capture device performs the 3D imaging procedure, suchas the structured-light 3D scanning procedure, including attempting todetermine the distance of the object (wound surface) located near thecenter of the view (step S404). If the image capture device successfullycaptures the 3D data (“yes” in step S408), the failure count is storedand the process returns. As a part of the successful capture process,the image capture device analyzes the captured data to generate a 3Dconstruction of the wound.

If the image capture device is unable to successfully capture the 3Ddata (“no” in step S408), which may be due to its inability todetermines the distance of the wound surface or due to other reasons,the image capture device increments the failure count (step S406), andrepeats the 3D data capture (step S404). In one embodiment, the imagecapture device emits a sound or displays an indication on the previewscreen and automatically restarts the scan, so the user is informed of afailed data capture attempt. In another embodiment, the image capturedevice may display an indication on the preview screen about a faileddata capture attempt and waits for the user to press a button to restartthe scan. In either embodiment, the user may slightly move the aim ofthe 3D scanner when a failure indication is displayed to improve thecapture condition.

The number of failures during the 3D data capture step S104 serves as anindication of whether the wound has a wet surface making 3D data capturedifficult. A relatively high number of failures indicates presents ofwet surface.

After the image capture device successfully captures the 3D image dataand analyzes the data to generate a 3D construction of the wound (stepS104 returns successfully), the image capture device emits anillumination light of the second intensity (step S106), and captures a2D image of the wound using the 2D camera 13 (step S108). The secondlight intensity is preferably lower than the medium or default lightintensity level of the light source 13 a, or at the medium or defaultlevel. This step may include capturing multiple images at differentexposure levels using the high dynamic range (HDR) mode of the 2Dcamera, so that the multiple images may be processed to generate an HDRimage. HDR techniques are generally known in the art. Alternatively, the2D image capture step may be repeated multiple times using differentillumination light intensities to capture the images with differentdynamic range and thus a wider dynamic range of image can be generated.The 2D image obtained in step S108 is used for wound assessment in stepS118.

In an alternative embodiment, if the number of failures during 3D datacapture does not exceed a predefined threshold T1 (e.g., 5 failures),then using HDR techniques is optional in the 2D image capture step S108.I.e., in this alternative embodiment, in step S108, image capture forHDR processing is preferably performed when the wound potentially has awet surface. This can simplify the process when the wound does not havea wet surface.

Then, if the number of failures during 3D data capture exceeds thepredefined threshold T1 (“yes” in step S110), the image capture deviceemits an illumination light of the third intensity (step S112), andcaptures a 2D image of the wound using the 2D camera 13 (step S114). Thethird light intensity is higher than the medium or default lightintensity level of the light source 13 a. In one embodiment, the thirdillumination light is a photographic flash light generated by the lightsource 13 a.

The image capture device then performs a process of determining themoisture level of the wound using the 2D image captured under the third(higher) illumination light intensity (step S116). The details of thisstep are shown in FIG. 5.

First, a preliminary step of wound area detection and extraction isperformed (not shown in FIG. 5). The detection may be based on colorand/or texture. This step may be done using the 2D image captures underthe third illumination light intensity in step S114, or preferably usingthe 2D image captures under the second illumination light intensity instep S108. The steps shown in FIG. 5 are performed on the extractedwould area of the 2D image.

In the extracted wound area, highly saturated pixels are set to white(step S502). This may be done by pixel value clustering on a color imageor binarization on a grayscale image. For a color image, clustering isperformed on the color values of the pixels to classify the color valuesinto two or more classes (using any suitable clustering algorithm), andpixel values that belong to the class that is the closest to the whitecolor value are changed to white.

After setting some pixels to white, connected component analysis isperformed to extract white connected components in the image, where eachwhite connected component is a group of white pixels that are connectedto each other (step S504). Step S504 optionally includes performing anerosion operation so that small connected components (which are likelyto be noise) are eliminated, followed by a dilation operation so thatclosely adjacent connected components are connected together.

The number of white connected components (Nw) is then counted, and thesize (i.e. area) of each white connected component is calculated (stepS506). If the number of white connected components Nw is less than apredefined threshold T2 (for example, T2=3), and the ratio of the mediansize of the white connected components to the size (i.e. width*height)of the wound area is greater than a predefined threshold T3 (forexample, T3=20%) (“yes” in step S508), then the moisture level of thewound is determined to be low (step S514).

Otherwise (“no” in step S508), the centroid location of each whiteconnected component is calculated, and the spatial distribution of thecentroids is calculated (step S510). If the standard deviation of thespatial distribution is greater than a predefined threshold T4 (forexample, T4=0.5), and the number of white connected components Nw isgreater than another predefined threshold T5 (for example, T5=10) (“yes”in step S512), then the moisture level of the wound is determined to behigh (step S516). Otherwise (“no” in step S512), the moisture level ofthe wound is determined to be medium (step S518). This concludes themoisture level assessment process S116.

It can be seen that the algorithm in step S116 (FIG. 5) is based on theobservation that under a more intense illumination light, wet areas willbe shiny and hence have a relatively large number of bright or saturatedpixels. In addition, the bright or saturated pixels are scatted due tothe uneven wound surface.

Referring back to FIG. 3, wound assessment is performed using the 3Dimage captured in step S104 and the 2D image captured at the second(lower) illumination light intensity in step S108 (step S118). Woundassessment may include measuring the length, width, depth, volume,color, texture, etc. of the wound. Any suitable methods may be used toperform the wound assessment. Then, the wound assessment results,including the results obtained from step S118 and the moisture levelassessment obtained from step S116, are registered in a database alongwith other information such as the patient's name, the nurse's name,date and time, etc.

FIG. 6 schematically illustrates a wound image capture and woundassessment process according to a second embodiment of the presentinvention. This process is identical to the process of the firstembodiment (FIG. 3) except: (1) The 3D image capture step S104A in FIG.6 is similar to step S104 (FIG. 4) but the failure count is notnecessary; and (2) step S110 in FIG. 3 is eliminated, i.e., steps S112,S114 and S116 are always performed regardless of the failure count.

FIG. 7 schematically illustrates a wound image capture and woundassessment process according to a third embodiment of the presentinvention. This process is identical to the process of the secondembodiment (FIG. 6) except: (1) The 3D image capture step (S102, S104A)is eliminated; and (2) the wound assessment step S118A in FIG. 7 issimilar to step S118 in FIGS. 3 and 6 but does not use 3D image data.

In embodiments of the present invention, the third illumination lightintensity is always higher than the second illumination light intensityand the first illumination light intensity (if it is used). A commonfeature of the first, second and third embodiments is that they use arelatively high illumination light intensity (which may be a flash) toacquire a 2D image containing a large number of saturated pixels toperform moisture level assessment, and use a relatively low illuminationlight intensity to acquire a 2D image for other aspects of woundassessment such as size, color, etc.

The wound image capture and wound assessment processes of the first tothird embodiments of the present invention are implemented using theimage capture device 1. Various steps of the processes, such as theemitting of the illumination lights (steps S102, S106, S112) and captureof the 3D and 2D images (steps S104, S108 and S114), may requireinitiation by the user, and the image capture device may display variousprompts on the user interface screen 15, 16 or prompt the user toperform the next step.

It will be apparent to those skilled in the art that variousmodification and variations can be made in the wound image capture andassessment method and related apparatus of the present invention withoutdeparting from the spirit or scope of the invention. Thus, it isintended that the present invention cover modifications and variationsthat come within the scope of the appended claims and their equivalents.

What is claimed is:
 1. A method implemented in a system including animage capture device for assessing a wound of a patient, comprising: (a)capturing a first image of a part of the patient's body which containsthe wound, the part of the patient's body being illuminated by a lightof a first intensity during the capture; (b) capturing a second image ofthe part of the patient's body, the part of the patient's body beingilluminated by a light of a second intensity during the capture, thesecond intensity being stronger than the first intensity; (c) estimatinga moisture level of the wound by analyzing the second image, including:(c1) extracting a wound area from the second image; (c2) extractingwhite connected components in the wound area, wherein each whiteconnected component is a group of white pixels in the wound area thatare connected to each other; and (c3) estimating the moisture level ofthe wound based on the extracted white connected components; (d)assessing the wound by analyzing the first image, including measuring atleast one of a size, texture and color of the wound; and (e) registeringassessment results of the wound, wherein the assessment result includesthe estimated moisture level and the at least one of the size, textureand color of the wound.
 2. The method of claim 1, wherein step (c2)includes: either performing a clustering operation on color values ofpixels of the wound area to classify the color values into two or moreclasses and setting color values of pixel that belong to a class that isthe closest to a white color value to white, or binarizing a grayscaleversion of the wound area of second image.
 3. The method of claim 1,wherein step (c3) includes estimating the moisture level of the woundbased on a number of the white connected components, sizes of the whiteconnected components, and a distribution of locations of the whiteconnected components.
 4. The method of claim 1, wherein step (c3)includes: when a number of the white connected components is less than afirst predefined threshold and a ratio of a median size of the whiteconnected components to a size of the wound area is greater than asecond predefined threshold, the moisture level of the wound isestimated to be low; and when the moisture level of the wound is notestimated to be low, and when a standard deviation of a spatialdistribution of centroids locations of the white connected components isgreater than a third predefined threshold and the number of whiteconnected components is greater than a fourth predefined threshold, themoisture level of the wound is estimated to be high.
 5. The method ofclaim 1, wherein the light of the second intensity is a photographicflash light.
 6. The method of claim 1, further comprising: (f) capturinga three-dimensional image of the part of the patient's body, the part ofthe patient's body being illuminated by a light of a third intensityduring the capture, the third intensity being weaker than the secondintensity; wherein in step (d), the assessing of the wound includesanalyzing the three-dimensional image and includes measuring a depth ofthe wound.
 7. The method of claim 6, wherein step (f) is performed usinga structured-light 3D scanner.
 8. The method of claim 1, wherein step(a) includes capturing multiple images at different exposure levels andprocessing the multiple images to construct the first image which is ahigh dynamic range image.
 9. A method implemented in a system includingan image capture device for assessing a wound of a patient, comprising:capturing a first image of a part of the patient's body which containsthe wound, the part of the patient's body being illuminated by a lightof a first intensity during the capture; capturing a second image of thepart of the patient's body, the part of the patient's body beingilluminated by a light of a second intensity during the capture, thesecond intensity being stronger than the first intensity; estimating amoisture level of the wound by analyzing the second image; assessing thewound by analyzing the first image, including measuring at least one ofa size, texture and color of the wound; using a structured-light 3Dscanner, capturing a three-dimensional image of the part of thepatient's body, the part of the patient's body being illuminated by alight of a third intensity during the capture, the third intensity beingweaker than the second intensity, including: attempting to capture imagedata of the part of the patient's body using the 3D scanner, includingmeasuring a distance of a surface of the wound; when the attempt isunsuccessful, incrementing a failure count and repeating the attemptingstep; and when the attempt is successful, constructing athree-dimensional image of the part of the patient's body and storingthe failure count; further assessing the wound by analyzing thethree-dimensional image and measuring a depth of the wound; andregistering assessment results of the wound, wherein the assessmentresult includes the estimated moisture level, the depth, and the atleast one of the size, texture and color of the wound.
 10. An imagecapture device comprising: a 2D camera for capturing two-dimensionalimages; a 3D camera for capturing three-dimensional images; a userinterface screen for displaying images and information and forinteracting with a user; a processor; and a computer usablenon-transitory memory having a computer readable program code embeddedtherein which is configured to be executed by the processor to controlthe 2D camera, the 3D camera, and the user interface, and to perform aprocess of assessing a wound of a patient, the process comprising: (a)capturing a first image of a part of the patient's body which containsthe wound, the part of the patient's body being illuminated by a lightof a first intensity during the capture; (b) capturing a second image ofthe part of the patient's body, the part of the patient's body beingilluminated by a light of a second intensity during the capture, thesecond intensity being stronger than the first intensity; (c) estimatinga moisture level of the wound by analyzing the second image, including:(c1) extracting a wound area from the second image; (c2) extractingwhite connected components in the wound area, wherein each whiteconnected component is a group of white pixels in the wound area thatare connected to each other; and (c3) estimating the moisture level ofthe wound based on the extracted white connected components; (d)assessing the wound by analyzing the first image, including measuring atleast one of a size, texture and color of the wound; and (e) registeringassessment results of the wound, wherein the assessment result includesthe estimated moisture level and the at least one of the size, textureand color of the wound.
 11. The image capture device of claim 10,wherein step (c2) includes: either performing a clustering operation oncolor values of pixels of the wound area to classify the color valuesinto two or more classes and setting color values of pixel that belongto a class that is the closest to a white color value to white, orbinarizing a grayscale version of the wound area of second image. 12.The image capture device of claim 10, wherein step (c3) includesestimating the moisture level of the wound based on a number of thewhite connected components, sizes of the white connected components, anda distribution of locations of the white connected components.
 13. Theimage capture device of claim 10, wherein step (c3) includes: when anumber of the white connected components is less than a first predefinedthreshold and a ratio of a median size of the white connected componentsto a size of the wound area is greater than a second predefinedthreshold, the moisture level of the wound is estimated to be low; andwhen the moisture level of the wound is not estimated to be low, andwhen a standard deviation of a spatial distribution of centroidslocations of the white connected components is greater than a thirdpredefined threshold and the number of white connected components isgreater than a fourth predefined threshold, the moisture level of thewound is estimated to be high.
 14. The image capture device of claim 10,wherein the light of the second intensity is a photographic flash light.15. The image capture device of claim 10, wherein the process furthercomprises: (f) capturing a three-dimensional image of the part of thepatient's body, the part of the patient's body being illuminated by alight of a third intensity during the capture, the third intensity beingweaker than the second intensity; wherein in step (d), the assessing ofthe wound includes analyzing the three-dimensional image and includesmeasuring a depth of the wound.
 16. The image capture device of claim15, wherein step (f) is performed using a structured-light 3D scanner.17. The image capture device of claim 16, wherein step (f) includes:(f1) attempting to capture image data of the part of the patient's bodyusing the 3D scanner, including measuring a distance of a surface of thewound; (f2) when the attempt is unsuccessful, incrementing a failurecount and repeating step (f1); and (f3) when the attempt is successful,constructing a three-dimensional image of the part of the patient's bodyand storing the failure count.
 18. The image capture device of claim 10,wherein step (a) includes capturing multiple images at differentexposure levels and processing the multiple images to construct thefirst image which is a high dynamic range image.